Learning with data adaptive features

The cost-complexity pruning generates nested subtrees and selects the best one. However, its computational cost is large since it uses hold-out sample or crossvalidation. On the other hand, the pruning algorithms based on posterior calculations such as BIC (MDL) and MEP are faster, but they sometimes produce too big or small trees to yield poor generalization errors. In this paper, we propose an alternative pruning procedure which combines the ideas of the cost-complexity pruning and posterior calculation. The proposed algorithm uses only training samples, so that its computational cost is almost same as the other posterior-based algorithms, and at the same time yield stable results as the cost-complexity pruning. Moreover it can be applied for comparing non-nested trees, which is necessary for the BUMPing procedure. Empirical results show that the proposed algorithm performs similarly as the cost-complexity pruning in the standard situation and works better for BUMPing.